Constrained Markov control processes in Borel spaces: the discounted case
نویسندگان
چکیده
We consider constrained discounted-cost Markov control processes in Bore1 spaces, with unbounded costs. Conditions are given for the constrained problem to be solvable, and also equivalent to an equality-constrained (EC) 1' inear program. In addition, it is shown that there is no duality gap between EC and its dual program EC*, and that, under additional assumptions, also EC* is solvable, so that in fact the strong duality condition holds. Finally, a Farkaslike theorem is included, which gives necessary and sufficient conditions for the primal program EC to be consistent.
منابع مشابه
Compactness of the space of non-randomized policies in countable-state sequential decision processes
For sequential decision processes with countable state spaces, we prove compactness of the set of strategic measures corresponding to nonrandomized policies. For the Borel state case, this set may not be compact [14, p. 170] in spite of compactness of the set of strategic measures corresponding to all policies [17,2]. We use the compactness result from this paper to show the existence of optima...
متن کاملDiscrete-time Markov control processes with discounted unbounded costs: Optimality criteria
We consider discrete-time Markov control processes with Borel state and control spaces, unbounded costs per stage and not necessarily compact control constraint sets. The basic control problem we are concerned with is to minimize the infinite-horizon, expected total discounted cost. Under easily verifiable assumptions, we provide characterizations of the optimal cost function and optimal polici...
متن کاملApproximation and estimation in Markov control processes under a discounted criterion
We consider a class of discrete-time Markov control processes with Borel state and action spaces, and <−valued i.i.d. disturbances with unknown density ρ. Supposing possibly unbounded costs, and under standard continuity and compactness conditions, we combine suitable density estimation methods of ρ with approximation procedures of the optimal cost function, to construct asymptotically discount...
متن کاملFrom Infinite to Finite Programs: Explicit Error Bounds with Applications to Approximate Dynamic Programming
We consider linear programming (LP) problems in infinite dimensional spaces that are in general computationally intractable. Under suitable assumptions, we develop an approximation bridge from the infinite-dimensional LP to tractable finite convex programs in which the performance of the approximation is quantified explicitly. To this end, we adopt the recent developments in two areas of random...
متن کاملSufficiency of Markov Policies for Continuous-Time Markov Decision Processes and Solutions of Forward Kolmogorov Equation for Jump Markov Processes
In continuous-time Markov decision processes (CTMDPs) with Borel state and action spaces, unbounded transition rates, for an arbitrary policy, we construct a relaxed Markov policy such that the marginal distribution on the stateaction pairs at any time instant is the same for both the policies. This result implies the existence of a relaxed Markov policy that performs equally to an arbitrary po...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Math. Meth. of OR
دوره 52 شماره
صفحات -
تاریخ انتشار 2000